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1. OVERVIEW 


This report briefly describes an instructional guideline generation 
system developed during the present one-year contract for NTEC (Analysis 
of Requirements and Methodology for Decision Training in Operational 
Systems, Contract No. N61339-77-C-0005). The directions of improvement 
_ for the system will be identified in this report, and the development 
of a computerized instructional development aid for decision tasks will 
be suggested as a means for improving the system in identified directions. 
Such an aiding system will be overviewed and a simulated instructional 
guideline generation session produced by the aiding system will be presented. 
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2. INSTRUCTIONAL GUIDELINE GENERATION SYSTEM 


As part of the present one-year project for NTEC, a system for the 
generation of instructional guidelines was developed. The system provides 
a procedure to generate essential information to be incorporated in the 
production of instructional materials for any decision task subject to 
training in any specific instructional enviornment. Such information 
includes: | 


(1) Identification of decision tasks in the form-of objective 
_ hierarchies. | 


(2) Classification of decision tasks. 


(3) Identification of task-specific and basic instructional 
contents. 


(4) Selection of the most effective instructional method. 
(5) Selection of the most effective instructional media. 


Although the project has been initially aimed at developing a 
systematic procedure for providing instructional guidelines for decision 
tasks encountered in LAMPS ASW operation, a task- independent methodology 
was developed to underlie the design in order to make application of the 
system to other task domains subject to minimal modification effort. 
Such a methodology allows incorporation of general purpose decision 
training components into task specific decision training programs. This 
incorporation provides trainees with the basic skills required for task- 
specific decision making within a new domain. Furthermore, Since the 
characteristics of decision tasks within any specific domain are subject 
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to change with time, the methodology will allow development of more 
effective training programs even in the case where there is no possibility 
of transfering the trainees to other task domains. 


— The instructional guideline generation system is presented in 
Figure 2-1. The system contains two basic modules: Decision Task 
Classification Preprocessor and Decision Training Guideline Generator. 

The Decision Task Identification Component precedes the Decision Task 
Classification Preprocessor module which is composed of two sub-components: 
Decision Type Recognition. and Decision Task Classification. The Decision. — 
Training Guideline Generator module contains five sub-components: 
Instructional Guideline Generation, Cognitive Level Weighting, Instructional 
Method Selection, Method/Media Relevancy Detection, and Instructional 

Media Selection. The description of the eight components and their 
interactions as well as the input/output of the system is described in 

the remainder of this chapter. In what follows, attention must be paid 

to the fact that the implicit procedure defined by the system is being 
performed (operated) by an instructional technologist. 


Decision Task Identification receives any training task presented 
in the objective hierarchies and works as a filter which separates 
decision tasks from non-decision tasks. The specification of this 
component follows the definition of a decision task: | 


(1) The objective of a decision task is to select an alternative 
from a specified set of alternatives. 


(2) This selection may require the formulation of alternatives 
(problem structuring). 


(3) There is a lack of completely specified criteria for either 
alternative formulation or alternative selection. 
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DECISION TASK CLASSIFICATION PREPROCESSOR DECISION TRAINING GUIDELINE GENERATOR 
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The output of the component is the identification of the input 
task either as a decision task, which causes the task to enter the 
Decision Type Recognitive and Decision Task Classification components, 
or aS a non-decision task, which results the initiation of a conventional 
ISD procedure. | 


Decision Type Recognition receives a decision task and identifies 
it as Type 1, Type 2, or Type 3. Type 1 represents decision tasks with 
processes requiring only problem structuring; Type 2 includes all decision 
tasks concerning only action selection; Type 3 is composed of .all decision 
tasks which require both problem structuring and action selection. Since 
the processes involved in problem structuring are of different cognitive 
complexity than the ones involved in action selection, the training 
process for the tasks belonging to different decision types must be 
conducted differently. The information concerning the type of the 
decision task enters Decision Task Classification and Cognitive Level 
Weighting components. | 


Decision Task Classification is based on a procedure similar to 
that of discriminant analysis. To distinguish between classes of decision 
tasks, a collection of discriminating variables is selected that measures 
_ characteristics on which the classes are expected to differ, that is, 
training content requirements. A set of variables was designed to 
provide satisfactory discrimination for decision tasks with known class 
membership. The classes have been identified, and the classification of 
any new decision task with unknown membership can be accomplished. Thirty 
different classes constitute the set of possible classes relevant to 
decision training (Table 2-1). Since Type 3 decision tasks requires processes 
involved in both Type 1 and Type 2, the class of any Type 3 decision task 
can be presented by a combination of the classes identified for the 
corresponding Type 1 and Type 2 decision tasks, as indicated in Figure 2-2. 


TABLE 2-1 
DECISION TASK CLASSES 


DECISION 
CLASS 7 : 
NUMBER TYPE CLASS ATTRIBUTE 
1 1 SWRS Single attribute, sell defined, time relaxed, static 
2 1 SWRD Single attribute, well defined, time relaxed, dynamic 
3 ] SWPS Single attribute, sell defined, time pressure, static 
4 i SWPD Single attribute, well defined, time pressure, dynamic 
5 ] SARS Single attribute, ambiguous, relaxed, static 
6 | 1 SARD Single attribute, ambiguous, relaxed, dynamic 
/ 1 SAPS Single attribute, ambiguous, pressure, static 
g 1 SAPD Single attribute, ambiguous, pressure, dynamic 
9 1  MWRS Multiattribute, well defined, time relaxed, static 
10 ] MWRD Multiattribute, well defined, time relaxed, dynamic 
1 1 MWPS Multiattribute, well defined, time pressure, static 
12 ] MWPD Multiattribute, well defined, time pressure, dynamic — 
13 1 MARS Multiattribute, ambiguous, time relaxed, static 
14 1 MARD Multiattribute, ambiguous, time relaxed, dynamic 
15 1 MAPS Multiattribute, ambiguous, time pressure, static 
16 1 MAPD Multiattribute, ambiguous, time pressure, dynamic 
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CLASS 
NUMBER TYPE CLASS 


V7 
18 
19 
20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
30 
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DECISION 


RSEU 
PSUE 


RMMN 


PMMN 
RMMS 


‘-PMMX 


RMMR 
PMMR 
RLEX 
PLEX 
RHUR 


-PHUR 
RSAT | 


PSAT 


TABLE 2-1 (Continued) 


DECISION TASK CLASSES 


Time 
Time 
Time 
Time 
Time 
Time 
Time 
Time 
Time 
Time 
Time 
Time 


Time 


Time 


ATTRIBUTE 


relaxed, subjective expected utility 
pressure, subjective expected utility 


relaxed, Maxi-min utility 


pressure, Maxi-min utility 


relaxed, Maxi-max utility 
pressure, Maxi-max utility 
polaxeds Hanienas regret 
pressure, Mini-max regret 
relaxed, Lexicography 
pressure, Lexicography | 
relaxed, Hurwicz 

pressure, Hurwicz 

relaxed, satisfying 


pressure, satisfying 
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Decision Task 
Decision Type 


FIGURE 2-2 


DECISION TASK CLASSIFICATION | 
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The decision task and the information concerning its type enters 
Decision Task Classification component. The output of the component 
represents the class of the decision task which is sent to Instructional 
Content Generation component. : | 


Instruction Content Generation receives the information about 
the class of the decision task and generates the required instructional 
contents. Since a set of basic instructional contents is associated 
with each class, the process involved is a simple table look-up. The 
basic instructional content appears as part of the system's output which 
together with the task-specific information presented in decision trees, 
constitute required instructional content for the decision training 
program. | 


Cognitive Level Weighting receives the decision type and selects 
a cognitive level weight vector. The idea of cognitive level weight » 
vector was initiated by the fact that the training objectives for 
different tasks with unequal cognitive complexity can be accomplished 
differently depending on the instructional method employed. Such dependency 
between cognitive complexity and instructional method suggested a detailed 
analysis of cognitive levels involved in performance of each decision 
behavior. The result of the analysis was the identification of a cognitive 
level weight vector associated with each decision type (Table 2-2). Each 
vector presents the degree of importance of the six cognitive levels with 
respect to the corresponding decision type. The six cognitive levels 
include: knowledge, comprehension, application, analysis, synthesis, and 
evaluation. Since a weight vector is assigned to each decision type, the 
process involved is a simple table look-up. The selected cognitive level 
weight vector is sent to Instructional Method Selection component. 


Instructional Method Selection receives the cognitive level weight 
vector, incorporates it with the data in method/attribute matrix, and 
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Cognitive 
Level 


Decision 
Type 


Type | 
Type 2 


Type 3 


TABLE 2-2 


COGNITIVE LEVEL WEIGHT MATRIX 


Knowledge Comprehension Application Analysis Synthesis Evaluation 


g 0.1 0.2 0.5 0.2 é 
0.09 0.11 0.51 0.17 0.06 0.06 
0.04 0.11 0.36 0.33 0.13 0.03 


e: very small weight, can be approximated by zero 
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results in the most effective instructional method. The method/attribute 
matrix represents the judgmental evaluation of the degree of effectiveness 

of fourteen major instructional methods with respect to the six cognitive 
levels (Table 2-3). The judgmental data is provided by a training specialist 
and a psychologist. A weighted average procedure is used to caluclate 

a degree of effectiveness for each instructional method with respect to 

the decision task. The method with the highest degree of effectiveness 

is selected as the most effective instructional method and appears as 

part of the system's output. 


Method/Media Relevancy Detection receives the selected instructional 
method and, using method/media relevancy matrix, identifies the media relevant 
to the selected method. Since some instructional media are not applicable 

to a training environment. employing a specific instructional method, a study | 
of the relevancy of each of the twelve instructional media with respect to 
each instructional method was performed. The result of the study dichotomized 
the relevant and irrelevant media for each instructional method. These © 
results appear in the method/media relevancy matrix, Table 2-4. The relevant 
instructional media are identified by a reference to the method/media 
relevancy matrix, and the information is sent to the Instructional Media 
Selection component. | 


Instructional Media Selection receives the relevant media and, 
using media/attribute matrix and the weights of different attributes with 
respect to the training environment, selects the most effective.media. 
The degree of effectiveness of each instructional media with respect to | 
different attributes was judgmentally assessed by a training specialist. 
The results of this assessment appears in Table 2-5. The weight vector 
which identifies the degree of importance of different attributes of the 

‘instructional method with respect to the specific training environment, 
is provided by the system user instructional technologist. A weighted 


TABLE 2-3 
METHOD/ATTRIBUTE MATRIX 
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TABLE 2-4 
METHOD/MEDIA RELEVANCY MATRIX 
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(oral presentation/chalk 
board) 

Textbook: 

Workbook, Problem Sets 


Pictures, Slides, 
Transparencies 


Audiotapes 
Motion Pictures 


Television/Videotape 
Recording 


Programmed Text 
CAI 

Part-task Trainer 
Simulator 


Natural Objects 
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MEDIA/ATTRIBUTE MATRIX 


Instructional] 


Self 


TABLE 2-5 


~ Motion/Ski11 


Situational 
(fidelity) 
Control over 


Detail 
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average procedure is used to calculate the degree of effectiveness of each 
relevant instructional media with respect to the decision task. The media 
with the highest degree of effectiveness is selected as the most effective 
instructional media and appears as part of the system's output. 
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PROCEDURE FOR INSTRUCTIONAL GUIDELINE GENERATION 


The process of instructional guideline generation for decision tasks 
is compatible with conventional Instructional System Development (ISD). 
This process can be considered as a module, within a structured ISD program, 


for recognition and classification of decision tasks as well as selection 


of the most effective instructional methods, media, and contents for those 
tasks. The process of instructional guideline generation (i.e., use of 


the system) can be defined as a systematic execution of a series of steps: 


(1) Define objective hierarchies for the training tasks. As in 


the case of any ISD program, the process of instructional 


_ guideline generation starts with identification of objective 


hierarchies for the training tasks. 


(2) Recognize Decision Tasks. The objective hierarchies are 


processed by the decision task identification scheme to 
recognize the decision tasks involved in the training. 


(3) Identify type and class of the decision task. Using the 


(4) 


method described as Decision Task Classification Preprocessor 


the type and the class of each decision task are identified. 


Select the Instructional Contents. The instructional content 


corresponding to different attributes of the class of decision 
tasks are combined to construct the basic instructional 
contents. The basic instructional content includes: general 
definition, amplification, rules, pitfalls and limitations, 
interactions with other decision elements, and prerequisits 
for each instructional content section. In addition to basic © 
instructional content, the decision tree, representing each | 
decision task, is constructed to provide the required task- 
specific information for instructional content. 
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(5) Identify Cognitive Level Weights. The cognitive level weights | 
vector associated with the decision type is identified by a 


simple table look-up procedure. 


(6) Select the Most Effective Instructional Method. A linear 

| evaluation procedure is performed by a summation over the 
results of the multiplication of elements of the cognitive 
level weights vector by corresponding elements of each 
vector associated with different methods in the method/attribute 

relevancy matrix. This procedure provides an effective value 

assessment for each instructional method. The method with 
the highest effectiveness value is selected as the most 
effective instructional method for training the decision 
task under consideration. 


(7) Identify the Instructional Media Incompatible with the 
Selected Method. The vector associated with the selected 


instructional method in the method/media relevancy matrix 

is identified by a Simple table look-up. Those instructional 
media corresponding to zero-value entries in the vector are 
considered as the ones incompatible with the selected method. 


(8) Select the Most Effective Instructional Media. The degrees — 
of importance of attributes, associated with each media, 

with respect to the training environment is defined in a 
weight vector. Then, a linear evaluation procedure, similar 
to the one used in item 6, is performed to select the most 
effective instructional media. A summation is performed 
on the results of the multiplication of elements of the 
weight vector by corresponding elements of each vector 
associated with different media in the media/attribute matrix. 


The result of this summation will define the degree of 
effectiveness of the media if the media is compatible with 
the selected method. In the cases where the media is not 
compatible, the degree of effectiveness will be zero. The 
media with the highest degree of effectiveness will be 
selected as the most effective instructional method to be 
used with the selected instructional method in the specific 
instructional environment. | 
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4. DIMENSIONS FOR IMPROVEMENT 


The instructional guideline generation system provides a systematic 
procedure for incorporating a great deal of data into the process of 
selecting different components of a decision training program. These 
data are of great importance to any instructional system development 
and, theoretically, should all be considered in the development of any 
program. However, in practice, the lack of a systematic procedure for 
incorporating these data caused the instructional technologists to ignore 
a great portion of the data during the design process. The procedure of © 
instructional guideline generation, developed in this project, provides 
systematic means for the management of data and its effective utilization 
in the process of instructional system development. However, a more | 
effective utilization of the instructional guideline generation system 
requires more detailed look into three problems: 


(1) elicitation of more accurate judgmental data. 


(2) generation of a control structure to guide the user in 
executing different steps of the system. 


(3) development of a computational aid for calculations required 
in the system. 


A computerized instructional guideline generation system can result in 
substantial improvements in all three dimensions. 
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5: COMPUTERIZED INSTRUCTIONAL DEVELOPMENT AID 


The effectiveness of the instructional guideline generation system 
can be enhanced by improvement of the system in the three dimensions 
identified in the previous chapter. Computerization of the instructional 
guideline generation system can result in substantial improvements in 
all three dimensions. The result will be a computerized system to aid 
the process of instructional development. | 


The aiding system follows the design described in Chapter 2. It 
has the potential of working in two different modes: (1) development mode 
and (2) application mode. Development mode will provide access to judgmental 
data, stored in the system, in order to modify them by an adaptive procedure. 
The accuracy of each data element will be assessed by analysis of the 
guidelines generated for decision tasks used as a test case. The © 
inaccurate data will be modified to adapt the system's output to a 
desirable response. Application mode represents a man/machine interactive 
session through which the information concerning the characteristics of 
the training environment, as well as the input decision task, are elicited 
from the user instructional technologist and, based on that information, 
the instructional guidelines for the decision task are generated. During 
interactive sessions, the user will be guided by the sequence and content 
of a series of questions designed to elicit the required information. 
Furthermore, all necessary calculations will be performed automatically 
by the system. Such a system simplifies the process of instructional 
_ guideline generation into a series of judgmental evaluations of elementary 
attributes. While the process is highly simplified for the user, the 
system automatically incorporates all the information acquired from the 
user into the process which results in the most effective instructional 
method and media as well as the required instructional contents. 
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6. SIMULATED INSTRUCTIONAL DEVELOPMENT SESSION 


A prototype application mode session of the computerized 
instructional development aid is simulated in the folloiwng pages. The 
user responses have been underlined. 


3 Describe the characteristics of the instructional environment 
by assigning an important weight (between zero and one) to each of the 


following attributes. 


Tz Learner paced control on the instructional media. 
AT(1) = .7 | 


2, Instructional media to be self instructional. 
AT(2) = .8 | 


3. Motion of the instructional media. 
~AT(3) = .3 | 


4. Fidelity of the instructional media in terms of situational details. 
~ AT(4) = .8- | | 


5. Learner control over the instruction speed (motion). 
AT(5) = .6 


6. Flexibility of the media for instructional sequencing. 
AT(6) = .4 | 


7. Instruction readability. 
AT(7) = .3 | 
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Context creation. 


3s 


Convenience for development of testing and evaluation. 


6 


Low cost of instruction. 


3 


Simplicity of use and maintenance. 


Here 


9 


is the attribute list with the weights you assigned. 


0.70: Learner paced control on the instructional media. 
0.80: Instructional media to be self instructional. 

0.30: Motion of the instructional media. 
0.80: Fidelity of the instructional media in terms of situation 
0.60: Learner control over the instruction speed (motion). 
0.40: Flexibility of the media for instructional sequencing 
0.30: Instruction readability. | 
0.80: Context creation. 
Q.60: Convenience for development of testing and evaluation. | 
0.30: Low cost of instruction. 
0.90: Simplicity of use and maintenance. 
Options are Type 


Weights are satisfactory. 


Change weight of an attribute. 


AT(8) = 
9. 
AT(9) = 
10. 
AT(10) = 
11. 
AT(11) = 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
WEIGHT 
10. WEIGHT 
11. WEIGHT 
1. 
2. 
3. 


Display complete list. | d 
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OPTION = c 


Attribute number 


10. WEIGHT = 
New Weight - .9 


OPTION = q 


Here 


1. WEIGHT 
2. WEIGHT 
3. WEIGHT 
4. WEIGHT 
5. WEIGHT 
6. WEIGHT 
7. WEIGHT 
8. WEIGHT = 
9. WEIGHT 
10. WEIGHT 
11. WEIGHT 


Q. 


30: 


Low cost of instruction. 


is the attribute list with the weights you assigned 


com> en o> Tae <> De > a <at> a <a> Ee > a> ee > a 


./0: 
.80: 
. 30: 
. 80: 
.60: 
40: 
30: 
.80: 
.60: 
.50: 
902 


Learner paced control on the instructional media. 
Instructional media to be self instructional. 

Motion of the instructional media. 

Fidelity of the instructional media in terms of situation. 
Learner control over the instruction speed (motion). 
Flexibility of the media for instructional sequencing. 
Instruction readibility. 

Context creation. | 

Convenience for development of testing and evaluation. 
Low cost of instruction. 

Simplicity of use and maintenance. 


Throughout this consultation session the above importance weights will be 
considered to represent the elements of the instructional environment. To 


present a new environment: cancel the current session and start a new one. 


Press RETURN when ready to continue. 


Introduce the new instructional task by its instructional objective. 


Enter description: 


press the ESC key then RETURN. 
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Classify the Contact as U.S. Ship, Enemy Ship or Natural Object. 


Do you want to change or correct the description? Type yes/no. No 


What is the time pressure on course representation? 
Specify it by a time pressure factor between zero and one. 


Factor = .3 


Does the objective of the task include selection of an alternative from 
a set of alternatives? Yes | 


Is the set of alternatives given? Yes 


Is there a completely specified procedure for the alternative selection? 
No : | | 


Once the alternatives are given, is there any critical time pressure on 
the operator for fast alternative selection? Yes 


Which of the following decision rules is the most suitable one for the 
task? | 


Subjective Expected Utility (SEU) 
Mini-max loss (MMX) 

Mini-min loss (MMN) 

Mini-max regret (MMR) 
Lexicography (LEX) 

-Hurwicz (HUR) 

Satisficing (SAT) 
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Enter a number between 1 and 7? 2 


INSTRUCTIONAL GUIDELINES 


Instructional objective: classify the contact as U.S. Ship, Enemy 
Ship or Natural Object. 


The task to be taught is a decision task. 
Decision Type: Type 2 
Decision Class: 2-PMMX 


Instructional Content Must Include the Following Items: 
Collecting subjective utilities under time pressures. Translating outcomes - 
into utilities. Aggregating utilities. Relation of loss and utility _ 
concept of minimizing maximum loss. Effect of time pressure on quality. 
Concept of payoff tables. | 


The Best Three Suggested Instructional Methods are: 


1. Discussion-Individual Tutorial for which the best fies | 
instructional media are: 


1. Programmed Text 
2. CAI 
3. Lecture (oral presentation/chalkboard) | 


2. Individual Investiation (Research) for which the best three 
instructional media are: 


1. Natural Objects 


2. Textbook — 
3. Workbook, problem sets 
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om. Projects - Individual for which the best three instructional 


media are: 

1. Natural bojects 

2. Textbook 

3. Workbook, problem sets 


Press RETURN when ready to continue. 


